Skip to content

bug: type top_p as float in Completion.create#436

Open
anxkhn wants to merge 1 commit into
Portkey-AI:mainfrom
anxkhn:patch-7
Open

bug: type top_p as float in Completion.create#436
anxkhn wants to merge 1 commit into
Portkey-AI:mainfrom
anxkhn:patch-7

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 10, 2026

Copy link
Copy Markdown
**Title:** type `top_p` as float in `Completion.create` / `AsyncCompletion.create`

**Description:**
- In `portkey_ai/api_resources/apis/complete.py`, `Completion.create` and
  `AsyncCompletion.create` annotated `top_p` as `Union[bool, NotGiven]`.
- `top_p` is a nucleus-sampling probability (a float, typically `0 < top_p <= 1`),
  not a boolean. Change the annotation to `Union[float, NotGiven]` at both sites.
- This is annotation-only: the value is forwarded unchanged into the internal
  `stream_create` / `normal_create` helpers, so there is no runtime behavior change.
- Add an offline regression test
  (`tests/test_complete_top_p_annotation.py`) that introspects the sync and async
  `create` signatures and asserts the `top_p` annotation includes `float` and not
  `bool`. The test uses only `inspect` / `typing`, so it needs no API key or network.

**Motivation:**
The `bool` annotation is inconsistent with the rest of the SDK and with the
underlying OpenAI completions parameter, and it breaks static type checking for a
valid call:

- Everywhere else in the SDK `top_p` is typed as a float variant, for example
  `Union[float, NotGiven]` in `generation.py`, `Union[float, Omit]` /
  `Union[Optional[float], Omit]` in `chat_complete.py`, `assistants.py`,
  `threads.py`, and `responses.py`, and `Optional[float]` in the response/param
  models.
- The sibling sampling parameters in these same two signatures
  (`temperature`, `frequency_penalty`, `presence_penalty`) are already
  `Union[float, NotGiven]`, so `top_p` was the lone outlier.
- The vendored OpenAI completions signature types `top_p` as `Optional[float]`.

As a result, a correct call such as `client.completions.create(..., top_p=0.9)`
is flagged by type checkers (mypy/pyright) as an incompatible-type error, and
IDEs suggest `True` / `False` for a probability field. This change makes the two
public text-completions entry points consistent with the rest of the SDK and the
upstream OpenAI signature.

**Related Issues:**
None. Per CONTRIBUTING, small bug fixes may be raised without an issue.

Completion.create and AsyncCompletion.create annotated top_p as
Union[bool, NotGiven], but top_p is a nucleus-sampling probability
(a float). Every other top_p in the SDK and the underlying OpenAI
completions parameter is a float, and the sibling sampling params
(temperature, frequency_penalty, presence_penalty) in the same two
signatures are already Union[float, NotGiven].

The bool annotation made a valid call such as
completions.create(..., top_p=0.9) fail type checkers and led IDEs to
suggest True/False for a probability field. Change the annotation to
Union[float, NotGiven] at both sites. Annotation-only; no runtime
behavior changes.

Add an offline regression test asserting the top_p annotation includes
float (and not bool) for both the sync and async create signatures.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant